My thesis addresses the problem of planning under teammate behavior uncertainty by introducing the concept of intentional multiagent communication within ad hoc teams. In partially observable multiagent domains, agents much share information regarding aspects of the environment such that uncertainty is reduced across the team, permitting better coordination. Similarly, we consider how communication may be utilized within ad hoc teams to resolve behavioral uncertainty. Transmitting intentional messages allows agents to adjust predictions of a teammate's individual course of action. In short, an ad hoc agent coordinating with an unknown teammate can identify uncertainties within its own predictive model of teammate behavior then request the appropriate policy information, allowing the agent to adapt its personal plan. The main contribution of this work is the characterization of the interaction between learning, communication, and planning in ad hoc teams.

Within multiagent teams research, existing approaches commonly assume agents have perfect knowledge regarding the decision process guiding their teammates' actions. More recently, ad hoc teamwork was introduced to address situations where an agent must coordinate with a variety of potential teammates, including teammates with unknown behavior. This paper examines the communication of intentions for enhanced coordination between such agents. The proposed decision-theoretic approach examines the uncertainty within a model of an unfamiliar teammate, identifying policy information valuable to the collaborative effort. We characterize this capability through theoretical analysis of the computational requirements as well as empirical evaluation of a communicative agent coordinating with an unknown teammate in a variation of the multiagent pursuit domain.

Coordination with an unknown human teammate is a notable challenge for cooperative agents. Behavior of human players in games with cooperating AI agents is often sub-optimal and inconsistent leading to choreographed and limited cooperative scenarios in games. This paper considers the difficulty of cooperating with a teammate whose goal and corresponding behavior change periodically. Previous work uses Bayesian models for updating beliefs about cooperating agents based on observations. We describe belief models for on-line planning, discuss tuning in the presence of noisy observations, and demonstrate empirically its effectiveness in coordinating with inconsistent agents in a simple domain. Further work in this area promises to lead to techniques for more interesting cooperative AI in games.

Maintaining an accurate set of beliefs in a partially observable scenario, particularly with respect to other agents operating in the same space, is a vital aspect of multiagent planning. We analyze how the beliefs of an agent can be updated for fast adaptivity to changes in the behavior of an unknown teammate. The main contribution of this paper is the empirical evaluation of an agent cooperating with a teammate whose goals change periodically. We test our approach in a collaborative multiagent domain where identification of goals is necessary for successful completion. The belief revision technique we propose outperforms the traditional approach in a majority of test cases. Additionally, our results suggest the ability to approximate a higher level model by utilizing a belief distribution over a set of lower level behaviors, particularly when the belief update strategy identifies changes in the behavior in a responsive manner.

Many scenarios require that robots work together as a team in order to effectively accomplish their tasks.  However, pre-coordinating these teams may not always be possible given the growing number of companies and research labs creating these robots.  Therefore, it is desirable for robots to be able to reason about ad hoc teamwork and adapt to new teammates on the fly.  This paper adopts an approach of learning policies to cooperate with past teammates and reusing these policies to quickly adapt to the new teammates.  This approach is applied to the complex domain of robot soccer in the form of half field offense in the RoboCup simulated 2D league.  This paper represents a preliminary investigation into this domain and presents a promising approach for tackling this problem.